This article introduces an energy-based model that is adversarial regarding data: it minimizes the energy for a given data distribution (the positive samples) while maximizing the energy for another given data distribution (the negative or unlabeled samples). The model is especially instantiated with autoencoders where the energy, represented by the reconstruction error, provides a general distance measure for unknown data. The resulting neural network thus learns to reconstruct data from the first distribution while crushing data from the second distribution. This solution can handle different problems such as Positive and Unlabeled (PU) learning or covariate shift, especially with imbalanced data. Using autoencoders allows handling a large variety of data, such as images, text or even dialogues. Our experiments show the flexibility of the proposed approach in dealing with different types of data in different settings: images with CIFAR-10 and CIFAR-100 (not-in-training setting), text with Amazon reviews (PU learning) and dialogues with Facebook bAbI (next response classification and dialogue completion).

The paper proposes a classification technique using deep nets to deal with: (a) covariate shift (i.e., when the train and test data do not share the same distribution) (b) PU settings (i.e., when there are only positive and unlabeled datapoints are available). The key idea is to train an auto-encoder that reconstruct the positive instances with small error and the remaining instances (negative or unlabeled) with a large error (above a desired threshold). This structure forces the network to learn patterns that are intrinsic to the positive class (as opposed to features that are discriminative across different classes). The experiments highlight that the proposed method outperforms baselines across different tasks with different data types (image, short text, and dialogues). Overall, I enjoyed reading the paper.

This article introduces an energy-based model that is adversarial regarding data: it minimizes the energy for a given data distribution (the positive samples) while maximizing the energy for another given data distribution (the negative or unlabeled samples). The model is especially instantiated with autoencoders where the energy, represented by the reconstruction error, provides a general distance measure for unknown data. The resulting neural network thus learns to reconstruct data from the first distribution while crushing data from the second distribution. This solution can handle different problems such as Positive and Unlabeled (PU) learning or covariate shift, especially with imbalanced data. Using autoencoders allows handling a large variety of data, such as images, text or even dialogues.